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Abstract 

This article examines evidence of academic skill development and transfer related to the taking of a first year 
Inquiry-based seminar course designed to enhance a range of self directed learning skills and their 
transferability to other learning contexts. The study compares a sample of academic work from two groups of 
Social Sciences students, one comprised of students who had taken the Inquiry course and the other who had 
not. The student work consists of l) papers submitted by participants who were asked for the best paper they 
had written at university and 2) descriptive narratives provided by participants of the steps they took in 
researching and writing that paper. Qualitative and quantitative analysis by multiple raters using a blinded 
protocol was conducted. The results show both meaningfully higher paper and skill assessments for students 
who had taken the inquiry seminar and evidence of transfer of skills and strategy to other learning contexts, 
supporting the hypothesis that transfer of core skills occurs under particular learning conditions that can be 
fostered through course design and enhanced through specific pedagogical objectives. 
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Abstract 

This article examines evidence of academic skill development and transfer related to the 
taking of a first year Inquiry-based seminar course designed to enhance a range of self 
directed learning skills and their transferability to other learning contexts. The study 
compares a sample of academic work from two groups of Social Sciences students, one 
comprised of students who had taken the Inquiry course and the other who had not. The 
student work consists of 1) papers submitted by participants who were asked for the best 
paper they had written at university and 2) descriptive narratives provided by participants 
of the steps they took in researching and writing that paper. Qualitative and quantitative 
analysis by multiple raters using a blinded protocol was conducted. The results show both 
meaningfully higher paper and skill assessments for students who had taken the inquiry 
seminar and evidence of transfer of skills and strategy to other learning contexts, 
supporting the hypothesis that transfer of core skills occurs under particular learning 
conditions that can be fostered through course design and enhanced through specific 
pedagogical objectives. 
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Introduction 

The changing nature of the workforce, the information age, and new understandings in the 
science of learning are leading to a changing consciousness about the goals of higher 
education (National Research Council 2000: 4; Barnett 1999). Increasingly, the focus has 
moved from "teaching" to "learning" (Barr and Tagg 1995) and developing "active learners 
who seek to understand complex subject matter and are better prepared to transfer what 
they have learned to new problems and settings" (National Research Council 2000: 12). 
Educational institutions at all levels are introducing new courses and programs that focus 
on student centered learning. 


https://doi.org/l0.20429/ijsotl.2009.030209 


1 


Developing Useful and Transferable Skills 


This educational orientation has taken many forms: active learning, self directed learning 
(Knowles, 1975), intentional learning (Bereiterand Scardamalia 1989), discovery learning, 
experiential learning, cooperative learning, problem based learning, autonomous learning 
(Thomas and Rohwer 1986), learning to learn (Smith 1990), and lifelong learning (Knapper 
and Crosley 2000). But while these terms all have distinguishable technical meanings, they 
all suggest aspects of the same thing, the growing importance of student centered learning 
and learning-how-to-learn (Knapper 2004). This orientation implies the need to focus higher 
education more solidly in the development of what Ramsden (2003) calls "general aims and 
higher level abilities" including skills in self-directed learning, collaborative problem solving, 
and team building as well as the more traditional abilities of identifying, accessing, 
assimilating and communicating information. In the most general sense these skills are 
meant to prepare students for both enhanced success in their formal studies and a lifetime 
of learning. 

Many university educators assume that students learn skills as a by-product of the overt 
curriculum consisting primarily of knowledge-oriented classes (Appleby 2001), that learning 
in one subject area enhances ability to learn in another, and that there are generalisable 
learning outcomes that accrue from a collection of university courses, such as abilities to 
analyse or think critically. That is, there has been a broad assumption that any abilities 
that are developed will be transferable to other academic contexts and to real life situations 
(Sheperd 2000). However, the relatively extensive literature on transfer of learning (Billing 
2007) suggests that such assumptions about the natural accrual of useful academic skills 
and the ability to put them to general use may be unwarranted. Thus, the issue of skills 
transfer is highly significant to the idea of learning how to learn. 

In broadest terms, transfer of learning occurs when previously learned knowledge and skills 
affect the way new knowledge and skills are learned and performed (Cormier and Hagman 
1987). Stated another way, transfer is the process of using knowledge or skills acquired in 
one context in a new or varied context. Rather than simply generalizing across problem 
contexts, transfer implies domains and tasks that are similar but not the same (Alexander 
and Murphy 1999). Perkins and Salomon (1988, 1989, 1992) distinguish between 'near' and 
'far' transfer. Near transfer of learning takes place when the contexts are similar as, for 
example, driving one kind of car after learning in another. Far transfer refers to transfer 
when the context is more remote or alien, such as using skills learned in team sports in a 
business context. These are intuitive notions and represent end-points on a spectrum - 
many examples would fall somewhere between near and far. Research evidence has shown 
that when a new challenge is sufficiently similar to a previous one, differences can be 
handled without much conscious thought; near transfer of knowledge or skills is 
commonplace. However, in situations which are sufficiently different, there is an abundance 
of evidence that transfer of learning often does not occur as hoped or expected (Perkins 
and Solomon, 1992). 

Assuming transfer is achievable but not trivial, much work has been done investigating the 
underlying factors and conditions supporting transfer. Perkins and Solomon (1992) 
concluded that transfer is more likely under two different conditions: when there is diverse 
practice of routines (called low road transfer) and when learners mindfully and explicitly 
search for abstract connections and self monitor their learning (called high road transfer). 
Recent research carried out by Chiabura and Marinova (2005) indicates that skills transfer 
depends upon two broad factors: individual dimensions such as goal orientation and training 
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self-efficacy, and contextual factors such as supervision and peer support. In his 2000 book 
Key Skills: Teaching and Learning for Transfer, Shepherd suggests that a transfer-friendly 
pedagogy is characterized by active and student-centered learning in which students and 
instructors engage in critical dialogue and where students are engaged and evaluated on 
project work and build on past knowledge. Enhancing transfer is also presumed to involve 
making students aware of their skills, emphasizing the utility of skills to other contexts, and 
linking skills to each other. The National Research Council (2000) suggests that 
metacognitive activities such as self-ref lection and self-assessment support the transfer of 
learning into other realms and contexts. In a recent review of the evidence, Billing (2007) 
concludes that transfer is enhanced when a) learning focuses on general principles of 
reasoning and abstract concepts and principles, b) there is explicit recognition of skills and 
how problems resemble each other, and c) when learning occurs in a cooperative social 
context which favours developing explanations, provision of feedback, and expectations of 
individual responsibility for learning. 

This article focuses on research examining whether a first year seminar course designed to 
enhance a range of self-directed learning skills (Justice et al 2009) and their transferability 
to other learning contexts, in fact provided a set of skills that were useful and transferable 
to other academic settings. The evidence derives from a comparison of a sample of 
academic work from two groups of Social Sciences students, one comprised of students who 
had taken the course Inquiry 1SS3 and the other who had not. The student work consists of 
1) papers submitted by participants who were asked for the best paper they had written at 
university and 2) descriptive narratives provided by participants of the steps they took in 
researching and writing that paper. Qualitative and quantitative analysis by multiple Raters 
using a blinded protocol was conducted to explore the differences between students'skills 
and proclivities in researching and writing papers. While writing a paper may reflect 
generalized problems of researching and communicating effectively, each new paper arises 
in a new context, creates new challenges, requires new ways of expressing ideas, and 
requires finding the internal motivation to succeed at the task. As such, it is a complex task 
relying on many skills that we consider to be sufficiently 'far' that it reflects an example of 
meaningful transfer of learning from one context to another. 

The intervention - Inquiry 1SS3 

The Inquiry 1SS3 course design, learning exercises, assessment methods, and guidance 
given to instructors, are described in two previous publications (Justice, Warry, Cuneo, 
Inglis, Miller, Rice & Sammon 2002; Justice, Rice, Warry, Inglis, Miller & Sammon 2007). 
Inquiry 1SS3 was an experimental first year seminar for Social Science students at 
McMaster, a mid-size Canadian research-intensive university. Designed to be taken upon 
entering university, the main goal of the course was to prepare students to get the most 
out of their subsequent university courses and to build a set of tools for enhanced learning 
beyond university. More specifically, the course aimed at developing students' abilities to 
conduct sound and thoughtful research and to communicate process and results clearly. 

The vision was of developing 'inquirers' - people with enhanced academic research skills 
and habits for learning through the engaged pursuit of knowledge about matters that 
interest or effect them. 

The design of Inquiry 1SS3 assumed that for students to become inquirers involves the 
development of: 1) a set of academic and intellectual skills; 2) a general procedural 
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strategy for using these skills to pursue knowledge and; 3) a willingness or proclivity to 
use those skills and strategy independently in a variety of learning contexts. 

Inquiry 1SS3 aimed to develop a range of both general and specific academic and 
intellectual skills. General skills included deep and critical approaches to knowledge 
development, reflexive practice, collaboration, and academic engagement. More specific 
skills include those immediately relevant to conducting an inquiry and communicating. 

For example, students were asked to think about and practice reading and writing critically, 
formulating questions, thinking through research strategies, using research libraries and the 
Internet, evaluating information, and putting interpretations and conclusions together in 
thoughtful ways facilitative of communication. 


Students were encouraged to learn and engage in a procedural strategy called the inquiry 
cycle consisting of a progressive series of stages beginning with the formulation of a 
complex question and moving toward deeper understanding of the factors related to the 
question (Figure 1). 


Figure 1 . The Inquiry Cycle (adapted from Justice et al. 2002) 

Engaging a topic 

Taking __—- & building basic 

responsibility / knowledge ’"^4 

for learning Evaluating ' Developing a question 


t 


X 


Communicating new Self reflection Anticipating possible 

understandings & answers and determining 

Self evaluation relevant information 


\ 


Weig hing evidence and 
synthesizing understandings 




Identifying resources and 
gathering information 


Assessing 

information 


The thinking underlying the course pedagogy was constructivist; it was assumed that 
knowledge developed through active engagement leads to enhanced integration and 
internalization (Abdal-Flaqq 1998). The course was designed to create conditions in which 
students felt they were in charge of what they learned. The course was also designed 
around a substantive theme of social representation intended to allow opportunity for 
students to grapple intellectually with their own social experiences and thus to become 
personally engaged with a topic. Students actively worked together both to explore their 
topics and to consciously develop new capabilities and awareness of learning processes and 
inquiry strategies. It was assumed that these features of the learning environment would 
ultimately lead students to take on more responsibility for their own learning and to seek 
knowledge that fulfilled their own learning objectives. It was assumed that by developing a 
strategy for pursuing knowledge and understanding, and explicitly teaching that strategy in 
an environment meant to foster personal engagement, self-direction and the enjoyment of 
learning, academic skills would be more transferable to other circumstances 
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Methodology 

This study was part of a larger McMaster Innovative Learning Study which is briefly 
summarized below and more fully described in Justice, Rice and Warry (2009). The larger 
study compares skill levels and academic performance over time of students who had taken 
Inquiry 1SS3 when they began university with students who had not taken this course. The 
two samples were drawn from a five year population of Social Sciences students. From the 
potential pool of 6244 candidates, 54 students who had had taken Inquiry 1SS3 (referred to 
as 'Inquiry students' or 'Inquiry group') and 71 students who had not taken Inquiry 1SS3 
('Non-Inquiry students' or 'Comparison group') completed the study. Students were paid 
$100.00 to participate in the study and were not aware of the purpose of the study, only 
that it concerned innovative learning. 

During the process of this larger study participants were asked to submit a paper they had 
written while in university that they considered to be their best, irrespective of the content 
of the paper or the mark they received for it. At the same participants were asked to 
prepare a brief narrative regarding the steps they had taken in researching and writing the 
paper that they had submitted. The instructions asked them to be thoughtful and truthful 
about the details of the process and to write about their thoughts and actions from the time 
the paper was first assigned until it was handed in to their instructor. This paper deals with 
the analyses of these two components of the larger study. 

We are aware of the complexities of conducting research in an educational setting where it 
is not possible for ethical or practical reasons, nor desirable for pedagogical reasons, to 
randomly assign students to a course. The fact that students chose whether or not to take 
the course means that measured differences between Inquiry and Non-Inquiry students are 
potentially related to either the effects of the course or to pre-existing differences between 
groups related to the self-selection process. As a result, we tested for any pre-existing 
differences in all dimensions for which we have data and looked at the results of our 
findings to identify areas where we could test for non-course related difference between the 
two groups. As Table 1 shows there are no meaningful differences between the study 
groups in gender, age or high school performance, nor in level 1 grade average. Nor were 
there significant differences in self-reported parents' income, student loans or highest 
degree students' had earned at the time of study. 


Table 1. Comparison of variables between Inquiry and Non-Inquiry Students (from 
Justice et al 2009) 


Level 

Means 

Non-Inquiry 

Inquiry 

% Female 

82 

74 

Age 

19.2 

19.1 

HS English 
77.0 

79.3 

HS GPA 
79.5 

79.8 

Level 1 Avg. 
6.9 

7.0 

Total 

78 

19.2 

78.1 

79.6 

6.9 


It is difficult to directly answer the important question of whether students who chose ISS3 
were more academically motivated or not. However, any motivational difference is reflected 
in neither high school nor first-year university performance. Part of the larger study, 
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students also had completed a 140 item questionnaire related to university experiences and 
attitudes toward learning (Justice et al 2009). Comparisons revealed no differences between 
the two groups in areas which would suggest pre-existing motivational differences. We thus 
feel it is reasonable to consider having taken or not taken Inquiry 1SS3 the explanation for 
differential performances between study groups. 


Analysis of Student Papers 

To test whether participation in Inquiry 1SS3 had an effect on subsequent student 
performance, we removed any identifying information and randomly assigned papers to 
volunteer Raters. The Raters were professional educators from universities across Canada 
who were recruited using names from the Society for Teaching and Learning in Higher 
Education (STLHE) Networking Guide. Each Rater was sent up to 32 papers so that each 
paper was independently assessed four times. 1 

Raters assessed papers using a rubric (Appendix A) consisting of a global score and a rating 
on sixteen items using a five point scale. The rubric was structured around discerning the 
skills and strategy of the inquiry cycle (Figure 1) as well as making inferences about social 
awareness and approaches to learning. Raters were given a set of basic procedural 
instructions but we relied on their experience to discern the meaning and make judgments 
about the rubric items. 

We assumed that the number of years a student had attended the university would have an 
effect on many of the performance indicators that we were measuring. To allow for this we 
analysed results by university level as well as study group. Participants were divided into 
those who had completed fewer than 84 units (Lower Level) versus those who had 
completed more than 84 units (Upper Level). 

Based in analysis of variance, we calculated effect size, that is, a measure of the magnitude 
of the treatment effect independent of the size of the sample, using Cohen's d (Cohen, 
1988). Because we were testing for the effects of a single course and controlling for level 
with a relatively small sample, we expected to see low effect sizes and relatively high p. 
values. We decided to accept differences as meaningful if they met both conditions of 
having a minimum effect size of 0.2 (which Cohen considers to be a small effect) in 
combination with a reasonable probability value which, we judge to be O.2., an approach 
more fully elaborated in Justice et al (2009). 
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Component Name 

Variable 

1. Awareness of social issues 

Cultural issues awareness 


Political issues awareness 


Social differences awareness 

2. Deep and self-directed learning 
style 

Self-directed approach to learning 

Deep approach to learning 

3. Inquiry Cycle Strategy Indicators 

Focussed around question 

Flypotheses to focus research 

Discusses limitations 

Evidence of self-reflection 

4. Sound writing indicators 

Clearly written 

Well organised 

Critical thinking 

Synthesized to coherent whole 

Proof-read & copy-edited 

5. Sound evidence and well 
referenced 

Academically sound evidence 

Correctly referenced 


A factor analysis of the questions in the rubric (not including "Global Score") resulted in 
the five components presented in Table 3. 

Table 3. Factor Analysis 


An analysis of variance was carried out using the five components against group and level. 
Two-way interactions were not meaningful and were removed from the model. Results are 
shown in Table 4. The top half of the table shows differences between lower level students 
(Lower) and upper level students (Upper), the bottom half of the table shows differences 
between students who had taken Inquiry 1SS3 (Inquiry) and the comparisons (Non-Inq). 
Bolded numbers are considered meaningfully higher. Negative effect size numbers indicate 
positive results for upper level and the Inquiry group respectively. 


Table 4. Effect sizes and F-values by Group and by Level, Component Results _ 

Mean Estimates Effect ANOVA 

Component Source size F p. 



Level 

Lower 

Upper 




Global score 


70.3 

77.5 

-1.225 

49.255 

.000 

Awareness of social issues 


69.2 

75.1 

-.664 

14.461 

.000 

Inquiry Cycle Strategy Indicators 


55.9 

60.4 

-.534 

9.348 

.002 

Sound writing indicators 


67.8 

76.7 

-.971 

30.953 

.000 

Deep and self-directed learning style 


62.7 

71.4 

-.635 

18.271 

.000 
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Sound evidence and well referenced 


67.1 

74.6 

-.699 

16.038 

.000 


Group 

Non-Inq 

Inquiry 




Global score 


72.7 

75.1 

-.422 

3.274 

.071 

Awareness of social issues 


71.2 

73.1 

-.213 

.835 

.361 

Inquiry Cycle Strategy Indicators 


56.3 

60.1 

-.454 

3.786 

.052 

Sound writing indicators 


70.1 

74.3 

-.462 

3.928 

.048 

Deep and self-directed learning style 


63.5 

71.2 

-.549 

6.648 

.010 

Sound evidence and well referenced 


68.8 

72.8 

-.378 

2.631 

.106 


As one would expect, upper level students scored meaningfully higher than lower level, 
(with moderate to large effect sizes in all factors), suggesting that time spent at university 
(an average of two full years between Upper and Lower) improved students' skills in 
assembling evidence and writing a good paper. 

The results also show meaningfully higher assessments for students who had taken Inquiry 
1SS3 compared to those who had not. All mean estimates except "Awareness of social 
issues" are higher for the Inquiry group and most differences are meaningful within the 
effect size and significance levels chosen for this study. The effect sizes are small to 
moderate suggesting that having taken Inquiry 1SS3 also improved students' skills in 
assembling evidence and writing a good paper, though to a lesser extent than two full 
years of university. 

The results were considered sufficiently robust after testing random subsets of the data 
and finding similar results to further investigate the individual variables making up the 
components. The results of this analysis of variance and the effect size calculations are 
shown in Table 5. Again, the top half of the table shows differences between lower level 
students (Lower) and upper level students (Upper), the bottom half of the table shows 
differences between students who had taken Inquiry 1SS3 (Inquiry) and the comparisons 
(Non-Inq). Bolded numbers are considered meaningfully higher. Negative effect size 
numbers indicate positive results for upper level and the Inquiry group respectively. 


Table 5. Effect sizes and F-values by Group and by Level, Variable Results 


Rating Variable 



Mean 

Estimates 

Effect 

size 

AN OVA 

Component 

Source 



F p. 



Level 

Lower 

Upper 



Global score 



70.3 

77.5 

-1.225 

49.255 .000 

Social differences awareness 

1 


68.1 

77.1 

-.898 

26.479 .000 

Cultural issues awareness 

1 


71.1 

75.3 

-.399 

5.227 .023 

Political issues awareness 

1 


68.5 

72.8 

-.400 

5.246 .023 

Deep learner 

2 


58.5 

67.7 

-.817 

21.887 .000 

Self-directed learner 

2 


65.5 

73.0 

-.732 

17.585 .000 

Discusses limitations 

3 


42.0 

49.3 

-.729 

17.438 .000 

Evidence of self-reflection 

3 


54.1 

63.1 

-.719 

16.984 .000 
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Focused around question 

3 

70.7 

71.0 

-.024 

.020 

.889 

Hypotheses to focus research 

3 

56.9 

58.3 

-.124 

.505 

.478 

Clearly written 

4 

72.5 

80.7 

-.755 

18.705 

.000 

Critical thinking 

4 

64.0 

75.5 

-.939 

28.915 

.000 

Proof-read & copy-edited 

4 

61.9 

72.7 

-.867 

24.659 

.000 

Synthesized to coherent 
whole 

4 

68.9 

76.2 

-.615 

12.422 

.000 

Well organised 

4 

71.5 

78.4 

-.591 

11.455 

.001 

Academically sound evidence 

5 

69.2 

77.2 

-.722 

17.119 

.000 

Correctly referenced 

5 

65.0 

71.9 

-.530 

9.217 

.003 



Group Non-Inq 

Inquiry 




Global score 


72.7 

75.1 

-.422 

3.274 

.071 

Social differences awareness 

1 

72.8 

72.4 

.039 

.029 

.866 

Cultural issues awareness 

1 

72.2 

74.2 

-.181 

.606 

.437 

Political issues awareness 

1 

68.7 

72.7 

-.384 

2.711 

.100 

Deep learner 

2 

60.7 

65.5 

-.434 

3.472 

.063 

Self-directed learner 

2 

66.5 

72.0 

-.551 

5.583 

.019 

Discusses limitations 

3 

44.0 

47.4 

-.340 

2.125 

.146 

Evidence of self-reflection 

3 

57.6 

59.5 

-.154 

.435 

.510 

Focused around question 

3 

67.0 

74.7 

-.607 

6.778 

.010 

Hypotheses to focus research 

3 

56.5 

58.7 

-.184 

.626 

.429 

Clearly written 

4 

74.7 

78.5 

-.361 

2.396 

.122 

Critical thinking 

4 

68.1 

71.3 

-.262 

1.268 

.261 

Proof-read & copy-edited 

4 

64.8 

69.8 

-.413 

3.132 

.077 

Synthesized to coherent 
whole 

4 

70.9 

74.3 

-.294 

1.590 

.208 

Well organised 

4 

72.2 

77.7 

-.470 

4.070 

.044 

Academically sound evidence 

5 

71.2 

75.2 

-.370 

2.524 

.113 

Correctly referenced 

5 

66.5 

70.4 

-.304 

1.700 

.193 


Upper level students scored meaningfully higher than lower level in all but two variables. 
Upper level students were no more likely than lower level students to focus their research 
around a question or to develop hypotheses. Though having taken Inquiry 1SS3 is also not 
associated with an increase in the use of hypotheses to focus research, having taken the 
course is associated with the likelihood of using a question as the focus of the research 
process, one of the central aspects of the Inquiry learning cycle strategy (Figure 1). Though 
the individual variable results by group are not statistically significant in every case, the 
pattern of these results is compelling - in every variable but one Inquiry group scores are 
higher than those of the Non-Inquiry comparisons. The taking of Inquiry 1SS3 is associated 
with an overall improvement in papers written in other university courses as well as the 
specific use of procedures developed in that course. 
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Narrative Study 

The narratives that students provided describing the steps they had taken in researching 
and writing the paper were analyzed by three independent Raters (different than the Raters 
who reviewed the papers described in the previous section) who were blind to the study 
design and purpose of the research, and did not know whether the participants had taken 
the Inquiry course or not. Raters were trained using selected cases until their analytical 
results converged sufficiently. To assess the narratives, Raters used a structured coding 
sheet (Appendix B) developed to gage the degree to which students followed processes 
related to the inquiry cycle strategy - for example whether students focused their research 
around a question - and to document other aspects of the research procedures followed by 
students such as consultations with peers, teaching assistants and librarians. Raters were 
also asked to make several inferences about the student's learning style, for example the 
degree to which the narrative seemed to reflect a self-guided learner. 

Raters found several overall differences between Inquiry and Non-inquiry students in how 
(they said) they went about researching and writing their papers. Inquiry students were 
more likely to have described focussing their research around a question than Non-inquiry 
students. Inquiry students were also judged to be less likely to have described a superficial 
research strategy, less likely to have indicated a concern with grades in conducting the 
research, and less likely to have indicated the re-using of a paper from a previous course. 
Inquiry students also indicated spending greater amounts of time researching their paper 
than the comparative groups. 

Lower Level Inquiry students (though not upper level Inquiry students) indicated more 
critical assessment of evidence than Non-inquiry students and indicated making greater use 
of a system for synthesizing their research. Similarly, Lower Level Inquiry student 
narratives indicated more meaningfully engagement in the research process and more 
overall satisfaction with that process. Table 3 presents a summary of these process 
differences. 


Table 6. Summary of differences in researching and writing processes 


Process area 

Meaningful 
advantage for 
Inquiry students 

Meaningful 
advantage for 
upper levels 

Was meaningfully engaged 

Yes* 

- 

Focused research around a question (not a topic) 

Yes 

Yes 

Did not try to prove something already known 

— 

— 

Had a strategy for data collection 

— 

— 

Accessed academically sound evidence 

— 

Yes 

Critically assessed the evidence 

Yes* 

— 

Had a system for synthesizing the evidence 

Yes* 

— 

Proof-read /copy-edited the paper 

— 

Yes 

Was aware of the limitations of the process 

— 

-- 

Was satisfied with the process overall 

Yes* 

-- 

Was not concerned with grades 

Yes 

-- 

Did not re-use a paper 

Yes 

-- 
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Made statements critical of own work 

— 

— 

Did not indicate superficial strategy to please 

Yes 

— 

professor 



Spent more time on research 

Yes 

Yes 

Spent more time on writing 

-- 

Yes 


* difference exists only among lower level students 


At both levels, there were noteworthy differences between Inquiry and Non-inquiry students 
related to the diversity of resources students utilized to do their research and write their 
papers. Inquiry students indicated higher rates of resource consultation than the 
comparison students, suggesting either actual higher rates or an enhanced consciousness of 
this as an aspect of researching and writing a paper. Upper level Inquiry students report 
consulting the library, the internet, and their peers with greater frequency than their upper 
level Non-inquiry students. Lower Level Inquiry students report consulting professors with 
higher frequency than the Lower Level comparisons 

Overall, of sixteen coding themes that refer to procedure, there are notable group 
differences in nine. In five areas the difference is meaningful for all Inquiry students and in 
four areas the difference is meaningful for only lower level students. In the remaining seven 
areas there was no difference between Inquiry and Non-Inquiry students; however, in no 
case does the comparison group have higher score counts. 

In areas in which Raters were asked to make inferences about the student authors based on 
their narratives, Raters discerned several meaningful differences found between Inquiry and 
Non-inquiry students but only amongst the lower level participants. Inquiry students were 
judged to be more critical in their thinking, and more interested in cultural, political or social 
issues than Non-inquiry students. 


Discussion 

Several limitations in research design should be considered in interpreting the results of 
this study. As mentioned, students were not randomly assigned to study groups but chose 
whether or not to take Inquiry 1SS3, raising the only partially answerable question of 
whether students who chose ISS3 were more academically motivated or not. We collected 
no information on what participants took in university after first-year, nor on the types of 
papers submitted by participants. Also the study sample was relatively small and though 
participation was invited in a random fashion, participants chose to engage in the project 
for unknown reasons. Finally, the difficulty of measuring an educational effect against a 
complex background of possible causes is compounded by the relative size of the 
intervention being measured which was a single course out of a total of as many as forty 
courses completed and taken as long as five years previous to the study. 

The results of the analysis of the students' papers and narratives show that against a 
background of general improvement associated with time spent at university, Inquiry 
students had an advantage over students who had not taken the course. The analysis of 
student papers showed inquiry students demonstrated comparatively stronger use of a 
variety of skills related to sound writing and appeared to be more self-directed and deep 
learners in their approach to constructing their papers. The narrative study showed 
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differences between students who had taken Inquiry 1SS3 and those who had not in a 
number of 'attitudinal' areas. Inquiry students were judged to be have been more engaged 
and satisfied with the process of constructing the paper they submitted and less oriented 
to grades or to superficial strategies to please the professor. 

The narrative study findings suggest that taking the Inquiry seminar is associated with 
students using the inquiry cycle strategy to develop a coherent research and writing 
process. Both time spent in university and taking the inquiry seminar is associated with 
focusing research around a question as opposed to a thesis or topic. However, with the 
passage of time since taking the Inquiry course the use of this strategy and the general 
procedural advantage becomes less obvious. Many of the procedural differences are only 
obvious when comparing lower level students. This suggests the Inquiry course provided 
students with a set of skills and procedures generally developed in university, but provided 
it earlier in their studies. 

There is other evidence, both self reported in the narrative and confirmed by inference from 
the results of student's research and writing processes, that some aspects of the inquiry 
cycle strategy were transferred to other contexts. In the analysis of student papers, papers 
submitted by students who had taken Inquiry 1SS3 were judged to be more often built 
around research questions, more likely to discuss limitations, and more clearly written and 
well organised. Inquiry students distinguished themselves from the comparison group in 
the narrative study by more often describing focusing their research around a question, 
critically assessing evidence, and having a system for synthesising evidence. The Inquiry 
group's use of questions as a focus of research identified both in the procedure descriptions 
and papers themselves, is particularly compelling as evidence of transfer from Inquiry 1SS3 
as it is not only the cornerstone of the inquiry cycle strategy but is also one of the only 
variables in the analysis of the paper that did not 'naturally' improve with time spent in 
university. 

Comparing the results of the two studies, upper level students had a clear advantage over 
lower level students in the analysis of student papers, but they had a much more muted 
advantage in the narrative study. The pattern was opposite to that seen when examining 
the data by study group; Inquiry 1SS3 students were very clearly distinguishable from the 
comparisons in the narrative analysis but less so in the analysis of student papers. This 
suggests that Inquiry students more clearly took away from the course a theoretical or 
abstract sense of learning skills and processes. They were consequently better able or prone 
to articulate the procedures they knew. But knowledge of this process did not necessarily 
translate into better papers as judged by Raters. Time spent in university, on the other 
hand, was associated more clearly with greater skill and procedural prowess, but not 
necessarily the ability or predilection to describe those skills and procedures. 

Finally, though in the analysis of student papers the effect of Inquiry 1SS3 participation was 
as strong for upper level students as for lower; in the narrative study the Inquiry effect was 
stronger among lower level students. This may suggest that methodologies measuring 
performance may have more success in documenting skills transfer than those measuring 
more abstract procedural knowledge. 
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Conclusions 

The literature suggests there are critical components to curricula design that enhance 
knowledge and skills transfer to other contexts - a process integral to learning to learn. 
Transfer does not just happen -- it is not a natural bi-product of education; rather it is a 
process that needs to be consciously fostered through course design, and nurtured in the 
classroom. 

Inquiry 1SS3 was developed to prepare students to be more successful learners in their 
subsequent university courses and beyond. As such the design of the course considered the 
development of specific academic and intellectual skills and their transferability to other 
learning contexts. The design of Inquiry 1SS3 meets a number of the conditions identified 
by Chiabura and Marinova (2005) by providing the opportunity for individual attainment of 
personal educational goals and skills and the contextual support of individual supervision 
and peer support. Shepherd (2000) argues that this type of transfer-friendly pedagogy 
encourages students to take control of the learning environment, to place their own 
interests at the center of their learning and to enter into discussions about their academic 
work. Three aspects of the course were developed with the idea of enhancing the 
transferability of learning skills. First, the substantive theme of the course allowed a broad 
selection of possible individual inquiries related to personally relevant and engaging topics. 
Second, skills were developed in the context of a general strategy for independent learning, 
the inquiry cycle. Finally the learning environment, based in the self-directed pursuit of an 
individual inquiry with regular support and feedback by the instructor and other students, 
was designed to emphasize a reflective and critical approach to knowledge and conscious 
awareness of developing skills and their relevance to learning. 

This research focused on the larger picture of whether students transferred these skills to 
other learning situations by looking for evidence of enhanced skills and the use of the 
inquiry cycle strategy in papers written for other courses and in student accounts of how 
those papers were constructed. The study results provide evidence that the Inquiry learning 
process was transferred from the course. First, the enhanced use of questions as the focus 
of research and writing is the fundamental and distinctive marker of the inquiry cycle 
strategy. Second, it appears that to a greater degree than actual skill development the 
students who had taken Inquiry 1SS3 were conscious of what they were doing; they could 
describe and had a language for the steps they had taken which allowed them to express 
their research and writing in clear terms. By demonstrating an increased ability to write 
well, Inquiry students confirm that they can take skills developed in one course and use 
them in new situations. The assessments of the papers and the descriptions in the 
narratives supports the notion that Inquiry students transferred subject matter knowledge, 
learning strategies and an interest in what they are learning into new situations. This 
research thus supports the hypothesis that transfer of core skills occurs under particular 
learning conditions that can be fostered through course design and enhanced through 
specific pedagogical objectives. 
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1 Inter-rater reliability was assessed by examining a sum of all the rubric items except the 
global score. Box plots and other examinations of the between Rater differences were 
carried out. The results of a single Rater were eliminated. Missing data was replaced with 
the mean of the other three Raters for that participant. 


APPENDIX A: RUBRIC FOR 'BEST PAPER' RATERS 


Paper Assessment Form Paper ID # 

McMaster Innovative Learning Study Assessor's Name_ 

A. Global Score: 


1. All things considered, and with reference to the other papers, please provide a numerical grade out of 100: 


B. Analysis of Paper 

Strongly 
Agree (5) 

Generally 
Agree (4) 

Neutral, 
N/A or 
Cannot 
Say(3) 

Generally 

Disagree 

(2) 

Strongly 

Disagree 

(1) 

The paper is clearly written 

5 

4 

3 

2 

1 

The paper is well organised 

5 

4 

3 

2 

1 

The author has focussed the research around a 
question 

5 

4 

3 

2 

1 

The author used hypotheses to focus the research 

5 

4 

3 

2 

1 

The author used academically sound evidence 

5 

4 

3 

2 

1 

The author demonstrated critical thinking 

5 

4 

3 

2 

1 

The paper is correctly referenced 

5 

4 

3 

2 

1 

The paper is synthesized into a coherent whole 

5 

4 

3 

2 

1 

The paper is carefully proof-read and copy-edited 

5 

4 

3 

2 

1 

The author discusses limitations of the paper 

5 

4 

3 

2 

1 

The paper demonstrates evidence of self-reflection 

5 

4 

3 

2 

1 

C. Analysis of Learning Style 

Strongly 
Agree (5) 

Generally 
Agree (4) 

Neutral, 
N/A or 
Cannot 
Say(3) 

Generally 

Disagree 

(2) 

Strongly 

Disagree 

(1) 

The paper suggests the author is a self-directed 
learner 

5 

4 

3 

2 

1 

The paper suggests the author is a deep learner 

5 

4 

3 

2 

1 

D. Analysis of Awareness 

Strongly 
Agree (5) 

Generally 
Agree (4) 

Neutral, 
N/A or 
Cannot 
Say(3) 

Generally 

Disagree 

(2) 

Strongly 

Disagree 

(1) 

The paper demonstrates awareness of cultural issues 

5 

4 

3 

2 

1 

The paper demonstrates awareness of political issues 

5 

4 

3 

2 

1 

The paper demonstrates awareness of social 
differences 

5 

4 

3 

2 

1 


Appendix B. Scoring Sheet for Narrative Raters 


Rater initials:_ Case # 

McMaster Innovative Learning Study: Writing Process Assessment Form __ 

| A. Global Score: | ~A f~B fc p5 \~F 
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(4) 

(3) 

(2) 

(1) 

(0) 

All things considered, the author probably is in the 
following GPA range 

4 

3 

2 

1 

0 


B. The following items refer to the narrative 
submitted by the students (not to the best paper 
described in the narrative) 

Strongly 

Agree 

(4) 

Generally 

Agree 

(3) 

Generally 

Disagree 

(2) 

(1) 

Strongly 

Disagree 

The narrative was on topic (followed instructions) 

4 

3 

2 

1 


The narrative provided a thorough description of the 
process undertaken 

4 

3 

2 

1 


The narrative was clearly written / is readable* 

4 

3 

2 

1 



* Point form was allowed in the instructions 



Mentioned 

Not 

mentioned 

C. The following items refer to the 
process they used to construct their 
paper (as judged from what is 
described in the narrative) 

Strongly 

(4) 

Genera 

iiy 

Agree 

(3) 

Generally 

Disagree 

(2) 

Strongly 

Disagree 

(1) 

No basis for 
judging 

(0) 

The author seemed to be meaningfully 
engaged with the topic 

4 

3 

2 

1 

0 

The author focussed the research around a 
question (as opposed to a thesis or topic) 

4 

3 

2 

1 

0 

The author seems to have set out to prove 
something already known 

4 

3 

2 

1 

0 

The author seems to have had a plan for 
selecting the information that would be 
used (e.g. as opposed to developing the 
paper based on what was found ) 

4 

3 

2 

1 

0 

The author seems to have accessed 
academically sound evidence 

4 

3 

2 

1 

0 

The author seems to have critically 
assessed the evidence 

4 

3 

2 

1 

0 

The author seems to have used some 
system for synthesizing the research - 
(connecting ideas / constructing a 
resolution) 

4 

3 

2 

1 

0 

The author seems to have proof-read /copy- 
edited the paper 

4 

3 

2 

1 

0 

The author seems aware of the limitations 
of the process 

4 

3 

2 

1 

0 

The author seems to have been satisfied 
with the process undertaken 

4 

3 

2 

1 

0 

The author seems to have been concerned 
with grades 

4 

3 

2 

1 

0 

The author seems to have re-used a paper 
or had more than one purpose for the paper 

4 

3 

2 

1 

0 
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Mentioned 

Not 

mentioned 

C. The following items refer to the 

Strongly 

Genera 

Generally 

Strongly 

No basis for 
judging 

process they used to construct their 


iiy 

Disagree 

Disagree 

paper (as judged from what is 


Agree 




described in the narrative) 

(4) 

(3) 

(2) 

(1) 

(0) 

The author made statements critical** of 
their own work, paper such as "it is not the 
best paper I have ever written" 

4 

3 

2 

1 

0 

The narrative indicates a superficial strategy 
for doing well (such as putting in buzz 
words from the syllabus, ideas that the 
professor likes, finding out what the 
professor wants etc) 

4 

3 

2 

1 

0 


** Means negatively critical, as in criticizing, not critical as in analytical 


D. The following items refer to the time 
spent on the best paper 

Several 

weeks 

(4) 

A week 

or so 

(3) 

Few Days 

(2) 

Just a few 
hours 

(1) 

No basis for 
judging 
(0) 

Over what time period does it seem the 
author did the research 

4 

3 

2 

1 

0 

Over what time period does it seem the 
author wrote the paper 

4 

3 

2 

1 

0 


E. The following items refer to whom or where the author 
consulted during the process 

More than 

once 

(2) 

Once 

(1) 

Not 

mentioned 

(0) 

Peers 

2 

1 

0 

Teaching Assistants 

2 

1 

0 

Professors 

2 

1 

0 

University programs, e.g. writing clinic, peer helper program 

2 

1 

0 

The library 

2 

1 

0 

The Internet 

2 

1 

0 


F. The following items refer to the author 

Strongly 

Agree 

(4) 

Generally 

Agree 

(3) 

Generally 

Disagree 

(2) 

Strongly 

Disagree 

(1) 

It seems the author is a self-directed learner 

4 

3 

2 

1 

It seems the author is a deep learner 

4 

3 

2 

1 

It seems the author is a critical thinker 

4 

3 

2 

1 

It seems the author is capable of self reflection 

4 

3 

2 

1 

It seems the author is a clear communicator 

4 

3 

2 

1 
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It seems the author is interested in cultural,political, 
or social difference issues 

4 

3 

2 

1 

It seems the author enjoys learning 

4 

3 

2 

1 
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